Adaptive optics systems and methods for vitreoretinal surgery

ABSTRACT

The present disclosure provides an adaptive optics system including at least one active pixel sensor array that detects light and sends a signal to a processor. The adaptive optics system also includes a wavefront correction system including at least one wavefront control structure and the processor, and that executes instructions on the processor to produce a digital image in which at least one wavefront distortion in the light detected by the active pixel sensor array is partially or fully corrected. The disclosure also provides methods of using the adaptive optics system.

TECHNICAL FIELD

The present disclosure relates to vitreoretinal surgery and surgicalequipment, and more specifically, to an adaptive optics system toimprove a digital image during vitreoretinal surgery and associatedmethods.

BACKGROUND

Ophthalmic surgery is surgery performed on the eye or any part of theeye. Ophthalmic surgery saves and improves the vision of tens ofthousands of patients every year. However, given the sensitivity ofvision to even small changes in the eye and the minute and delicatenature of many eye structures, ophthalmic surgery is difficult toperform and the reduction of even minor or uncommon surgical errors ormodest improvements in accuracy of surgical techniques can make anenormous difference in the patient's vision after the surgery.

One type of ophthalmic surgery, vitreoretinal surgery, encompassesvarious delicate procedures involving internal portions of the eye, suchas the vitreous humor, the retina, epiretinal membranes, and internallimiting membrane. Different vitreoretinal surgical procedures are used,sometimes with lasers, to improve visual sensory performance in thetreatment of many eye diseases, including epimacular membrane, diabeticretinopathy, vitreous hemorrhage, macular hole, detached retina,vitreomacular traction syndrome, macular schisis, and complications ofcataract surgery, among others.

During ophthalmic surgery, such as vitreoretinal surgery, anophthalmologist typically uses a non-electronic, optical, surgicalmicroscope with oculars to view a magnified image of the eye undergoingsurgery. More recently, vitreoretinal surgeons may use an ocular-freedigital image system to aid visualization during vitreoretinal surgery.These systems may include a 3D high dynamic range (“HDR”) camera systemwith a pair of 2D complementary metal-oxide-semiconductor (CMOS) singlechip or three-chip sensors that allows the surgeon to view the retina ona display screen using polarized glasses, digital oculars or ahead-mounted display. The display screen provides relief from having toview the surgery using oculars and allows others in the operating roomto see exactly as the surgeon does. The system also allows for improvedimages under high magnification, and increased depth of field comparedto a conventional optical, analog surgical microscope, which allow forimproved visualization of the eye.

SUMMARY

The present disclosure provides an adaptive optics system that improvesa digital image during vitreoretinal surgery. The adaptive optics systemincludes at least one active pixel sensor array that detects light andsends a signal to a processor. The adaptive optics system also includesa wavefront correction system including at least one wavefront controlstructure and the processor, and that executes instructions on theprocessor to produce a digital image in which at least one wavefrontdistortion in the light detected by the active pixel sensor array ispartially or fully corrected.

The adaptive optics system and its methods of use may include thefollowing additional features: i) the system may include a plurality ofactive pixel sensor arrays; ii) the wavefront control structure mayinclude a spatial light modulator (SLM). a liquid crystal on siliconspatial light modulator (LCoS-SLM), a transmissive LCoS-SLM, areflective LCoS-SLM, a deformable mirror, or any combination thereof;iii) the wavefront control structure may include a phase-only SLM; iv)the digital image may be displayed on a digital display, a screen, ahead up display, a head mounted display, or any combination thereof; v)the system may be a component of NGENUITY® (Novartis AG Corp.,Switzerland); vi) the active pixel sensor array may be a CMOS monochromesensor, a 4K monochrome CMOS sensor, a 1080P monochrome CMOS sensor, orany combination thereof; vii) the system may include an on-chipprocessor on the active pixel sensor array that implements a region ofinterest (ROI) gain control; viii) the system may include anamplitude-only SLM to implement a ROI gain control on the active pixelsensor array; ix) the system may include an image reference system; x)the image reference system may include an image reference that is asurgical tool placed in an eye; xi) the surgical tool placed in the eyemay be a vitreous cutter, forceps, scissors, a pic, a scraper, a flexloop, a spatula, a micro-vitreoretinal (MVR) blade, a micro-cannula orany combination thereof.

The present disclosure further provides an adaptive optics system thatincludes a time sequential color system that includes a red lightsource, a green light source, and a blue light source that emit asequence of red, green, and blue light spaced over time. The system alsoincludes at least one active pixel sensor array that detects each of thered, green, and blue light, and sends a signal to the processor. Thesystem may include the following additional features: i) the timesequential color system may include a red light emitting diode (LED), agreen LED, and a blue LED pulsed in sequence at an aggregate rate of 180Hz or above; ii) time sequential color system may include a redsuperluminescent light emitting diode (SLED), a green SLED, and a blueSLED pulsed in sequence at an aggregate rate of 180 Hz or above; iii)the time sequential color system may be delivered by an endoilluminator;iv) the active pixel sensor array may be an active pixel sensor arraywithout a Bayer filter; v) the active pixel sensor array may capturesequentially a red image, a green image, and a blue image.

The present disclosure further provides a medical system that includes aprocessor, at least one active pixel sensor array that is coupled to theprocessor, a wavefront correction system that is coupled to theprocessor, an image reference system, and a memory medium that iscoupled to the processor. The memory medium includes instructions that,when executed by the processor, cause the medical system to utilize theimage reference system to determine a wavefront distortion of areflected wavefront of light reflected off an interior of an eye of apatient. The memory medium further includes instructions that, whenexecuted by the processor, cause the medical system to utilize thewavefront correction system to correct the wavefront distortion of thereflected wavefront of light reflected off the interior of the eye.

The present disclosure further provides a method of correcting awavefront distortion to improve a digital image by using an imagereference system to determine a wavefront distortion of a reflectedwavefront of light reflected off the interior of the eye; and using awavefront correction system to correct the wavefront distortion of thereflected wavefront of light reflected off the interior of eye. Thepresent disclosure also provides a method of eliminating lateral colorspread and improving color rendition to improve a digital image by usinga time sequential color system emitting a sequence of a red light, agreen light, and a blue light spaced over time to illuminate theinterior of the eye; detecting a red wavefront of light, a greenwavefront of light, and a blue wavefront of light reflected off theinterior of the eye using at least one active pixel sensor array withouta Bayer filter; capturing a red image, a green image, and a blue imageof the interior of the eye sequentially using the active pixel sensorarray without a Bayer filter; and integrating the red image, the greenimage, and the blue image in the viewer's visual cortex, or reformattingthe red image, the green image, and the blue image for an organiclight-emitting diode (OLED) display, to give a color image of theinterior of the eye with lateral color spread eliminated.

Aspects of the adaptive optics system and its methods of use may becombined with one another unless clearly mutually exclusive. Inaddition, the additional features of the adaptive optics system and itsassociated methods described above may also be combined with one anotherunless clearly mutually exclusive.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present disclosure and itsfeatures and advantages, reference is now made to the followingdescription, taken in conjunction with the accompanying drawings, whichare not to scale, in which like numerals refer to like features, and inwhich:

FIG. 1 is a schematic representation of an adaptive optics system,including a wavefront correction system, a time sequential color system,and two monochrome active pixel sensor arrays;

FIG. 2 is a schematic representation of a portion of an adaptive opticssystem, including a wavefront correction system and a time sequentialcolor system;

FIG. 3 is a schematic representation of an adaptive optics system as acomponent of a visualization system;

FIG. 4 is a schematic representation of an adaptive optics system as acomponent of the NGENUITY® 3D Visualization System (Novartis AG Corp.,Switzerland); FIG. 5 is a schematic representation of an adaptive opticssystem as a component of a camera head on a surgical microscope;

FIG. 6 is a schematic representation of an adaptive optics system,including an image reference system;

FIG. 7 is a flow diagram illustrating a method of correcting a wavefrontdistortion to improve a digital image for vitreoretinal surgery;

FIG. 8 is a flow diagram illustrating a method of eliminating lateralcolor spread in an adaptive optics system improve a digital image forvitreoretinal surgery;

FIG. 9 is a schematic representation of a computer system, including anadaptive optics system;

FIGS. 10A-10C are schematic representations of a medical system,including an adaptive optics system; and

FIG. 11 is a schematic representation of a medical system, including anadaptive optics system, a surgeon, and a patient.

DETAILED DESCRIPTION

The present disclosure provides systems including adaptive optics toimprove a digital image for vitreoretinal surgery and associatedmethods.

Vitreoretinal surgeons face unique challenges when visualizing theinternal portions of the eye. For example, any view obtained through thepatient's pupil is subject to optical aberration. Optical aberration maybe caused by eye diseases or prior surgery causing corneal asphericityor intraocular lens implants which lead to an aberrated image viewed bythe surgeon. Spherical aberration may be caused by dilation of thepupil, oblique viewing to visualize the peripheral retina, cataract,intraocular lenses, and corneal asphericity. Chromatic aberration, whichmay be lateral or axial, may be caused by the failure of the eye'soptical system or retinal visualization system to focus different colorsto the same focal point or plane. Aberration may interfere with theability of the surgeon to visualize the interior of the eye and makesurgery more difficult. In analog systems there are very limited ways tocorrect for the effect of aberrations, and many are simplyuncorrectable. However, digital visualization systems do allow forvarious corrective measures as described herein, which may improve theimage presented to the surgeon and others assisting with vitreoretinalsurgery.

In particular, the systems and methods disclosed herein may reduce theeffects of aberrations in a digital image of the eye seen by the surgeonand others. By doing so, the systems and methods may reduce aberrationand improve digital image resolution, digital image color rendition,digital image dynamic range, or any combinations thereof duringvisualization in any aspects of vitreoretinal surgery. Systems andmethods of the present disclosure may improve a digital image forvitreoretinal surgery as compared to current systems and methods byincluding an adaptive optics system that may reduce spherical aberrationor chromatic aberration, reduce aberration from defocus caused by myopiaand hyperopia, regular astigmatism, coma, or trefoil, eliminate lateralcolor spread, allow region of interest (“ROI”) gain control, or anycombinations thereof.

Current systems and methods for digital visualization duringvitreoretinal surgery do not include an adaptive optics system. Instead,they rely on a twin sensor high dynamic range camera system, where theeffects of aberration may manifest as wavefront distortion, leading toimage distortion, and reduced image quality. Wavefront distortion invitreoretinal surgery may also be caused by retinal visualizationsystems (for example, macular or wide-angle contact lenses, ornon-contact viewing systems such as BIOM® (OCULUS Surgical, Inc., USA)and ReSight® (Carl Zeiss Meditec AG, Germany)), or corneal asphericitythat may result from photorefractive keratectomy (PRK), laser-assistedin situ keratomileusis (LASIK), penetrating keratoplasty (PK), radialkeratotomy (RK), limbal relaxing incision (LRI), Descemet membraneendothelial keratoplasty (DMEK), Descemet' s stripping endothelialkeratoplasty (DSEK), anterior lamellar keratoplasty (ALK), inlays,trauma, keratoconus, or pterygium. Wavefront distortion in vitreoretinalsurgery may alternatively be caused by tilt due to oblique viewing angleto see the peripheral retina, a dilated pupil, a crystalline lens withor without a cataract, an intraocular lens (especially multifocal orextended depth of focus intraocular lenses), or corneal astigmatism inaphakic eyes. Current adaptive optics systems used in ophthalmicresearch are typically very high magnification, non-stereo,monochromatic, not real time, have a very small field of view, require aminimally-moving subject, and are not suitable for surgery. The adaptiveoptics system to improve a digital image for vitreoretinal surgery asdescribed herein may utilize broadband white light, work in real time,display stereo images, have no apparent latency, and provide the samemagnification as current surgical microscopes and state-of-the-artDigitally Assisted Vitreoretinal Surgery (“DAVS”) system NGENUITY®(Novartis AG Corp., Switzerland).

The adaptive optics system described herein may correct wavefrontdistortion to improve a digital image for vitreoretinal surgery byincluding a wavefront correction system. The wavefront correction systemmay include a wavefront control structure that controls and correctswavefront distortion resulting in reflected light with a correctedwavefront. The wavefront control structure may be a deformable mirror,where the surface of the mirror may be deformed to control and correctwavefront distortion. The wavefront control structure may also be aspatial light modulator (“SLM”). The SLM may be a pixelated device thatmodulates phase, or polarization of light waves. The SLM may be used forwavefront shaping by controlling the phase of incident light at eachpixel. SLMs have a much higher resolution compared to deformable mirrorsthereby enabling complex wavefront shaping, point spread functionengineering, and pixel engineering that may reduce the effects ofaberration. The wavefront control structure may be a phase-only SLM. Thewavefront control structure may also be a liquid crystal on silicon SLM(“LCoS-SLM”), may be a transmissive LCoS-SLM, or may be a reflectiveLCoS-SLM. Wavefront shaping by the SLM may correct wavefront distortionand reduce the effects of aberration, which may be spherical aberration,to improve a digital image.

In the adaptive optics system described herein, a wavefront correctionsystem may be utilized in conjunction with a time sequential colorsystem. The time sequential color system provides a red light source, agreen light source, and a blue light source, which may emit a sequenceof red, green, and blue light spaced over time. The red light, greenlight, and blue light sources may be pulsed in sequence at an aggregaterate above 180 Hz to avoid color flicker or color breakup. The timesequential color system provides red, green, and blue light, and mayilluminate the eye during surgery. The red light, green light, and bluelight sources may be delivered by an endoilluminator, which is afiberoptic probe inserted through the pars plana. Wavefront distortionfrom the red light, green light, and blue light delivered by theendoilluminator and reflected by the eye may be corrected by thewavefront correction system. The implementation of a wavefrontcorrection system and a time sequential color system in the sameadaptive optics system may allow different wavefront shaping ortransform for each of the red, green, and blue light sources, which mayreduce chromatic aberration in a digital image by focusing red, green,and blue colors to the same convergence point or image plane. This mayreduce lateral chromatic aberration, axial chromatic aberration, or acombination thereof.

Alternatively, the time sequential color system may utilize at least onelight source that is a color other than red, green, or blue. The timesequential color system may also utilize at least three light sourcesthat may be colors of different wavelengths.

The adaptive optics system to improve a digital image for vitreoretinalsurgery described herein may use an image reference to improve thedigital image of the eye. The image reference may be used in a similarway as a guide star is used in adaptive optics systems for astronomy.

Generally, an image reference may be a reference object with a knownshape that is placed in the eye. In the adaptive optics system describedherein, a surgical tool may be used as an image reference. The surgicaltool may be a vitreous cutter, forceps, scissors, a pic, a scraper, aflex loop, a spatula, a MVR blade, a micro-cannula or any combinationthereof. If the shape of the surgical tool is not known before use, anon-aberrated image of the surgical tool may be created outside of theeye before the surgical tool is placed in the eye. This may allow thesame surgical tool when placed internally in the eye during surgery tobe used as an image reference. The wavefront distortions in the viewpathway, which may be in reflected light from the endoilluminator, maybe corrected using the wavefront correction system to restore an imageof the image reference to its non-aberrated appearance, which may be thenon-aberrated image of the surgical tool. The light may come from a timesequential color system delivered by an endoilluminator. A processor mayanalyze the wavefront distortions in the digital image of the imagereference and send instructions to the wavefront correction system tocontrol and correct the wavefront distortions to improve the digitalimage of the image reference. The same instructions may be used tocontrol and correct the wavefront distortions in the light reflected offthe eye to improve the digital image of the eye. In this example, aShack-Hartmann wavefront sensor may not be utilized.

The adaptive optics system to improve a digital image for vitreoretinalsurgery described herein may include an active pixel sensor array, whichis a light sensor array that detects light and conveys the informationused to make a digital image. An active pixel sensor array may be anarray of light-capturing cells, typically with each cell representing apixel. A pixel in an active pixel sensor array may detect onlybrightness information, and no color information. A color filter calleda Bayer filter may be used to allow an active pixel sensor array tofunction as a color sensor. A Bayer filter applies a red, green, or bluefilter to each pixel, and may be placed in front of the active pixelsensor array. The Bayer filter allows the correct brightness and colorinformation to be calculated for each pixel, and may result in a colordigital image. The adaptive optics system described herein may include asingle active pixel sensor array. The active pixel sensor array mayinclude left and right channels to provide a 3D stereo image.Alternatively the adaptive optics system may include two active pixelsensor arrays, or may include multiple active pixel sensor arrays. A 3Dstereo image may also be provided using a pair of active pixel sensorarrays.

The active pixel sensor array in the adaptive optics system may includea Bayer filter, or may be an active pixel sensor array without a Bayerfilter. The Bayer filter may introduce lateral color spread into adigital image for vitreoretinal surgery, which may lead to decreasedcolor rendition. In the adaptive optics system described herein, theinclusion of a time sequential color system may allow the active pixelsensor array to capture sequentially a red image, a green image, and ablue image. The red image, the green image, and the blue image may beintegrated in the viewer's visual cortex to give a color image of theeye. Alternatively, the red image, the green image, and the blue imagemay be processed electronically by an image processor and reformattedfor an OLED display, digital oculars or a head-mounted display. This mayremove the need for a Bayer filter on the active pixel sensor array toobtain a color image. Therefore, the adaptive optics system describedherein may be used for color visualization using an active pixel sensorarray without a Bayer filter but with a time sequential color system,which may reduce lateral color spread on the active pixel sensor arrayand increase color rendition in a digital image for vitreoretinalsurgery.

The adaptive optics system for an improved digital image invitreoretinal surgery in the present disclosure may further include anon-chip processor on the active pixel sensor array, which may implementa region of interest (ROI) gain control to improve dynamic range. ROIgain control may also be implemented by including an additionalamplitude-only SLM to improve dynamic range. The amplitude-only SLM maybe positioned in a focal plane before the active pixel sensor array.

The systems and methods of the present disclosure may provide anycombination of numerous advantages over than what is typically providedby general visualization sensors for vitreoretinal surgery, including:(1) improving a digital image for vitreoretinal surgery by improvingimage resolution through the use of a wavefront correction system tocorrect wavefront distortion; (2) improving a digital image forvitreoretinal surgery by improving color rendition through the combineduse of a time sequential color system and an active pixel sensor arraywithout a Bayer filter; and (3) improving a digital image forvitreoretinal surgery by improving dynamic range through the use of anactive pixel sensor array with ROI gain control.

Referring now to FIG. 1, adaptive optics system 100 includes wavefrontcorrection system 110, time sequential color system 120 and active pixelsensor arrays 130 a and 130 b. Adaptive optics system 100 may correctwavefront distortion 140 to corrected wavefront 150 using wavefrontcorrection system 110 to improve image resolution. Wavefront correctionsystem 110 may include wavefront control structure 111. Wavefrontcontrol structure 111 may control and correct the phase of wavefrontdistortion 140. Wavefront control structure 111 may be a SLM, anLCoS-SLM, a reflective LCoS-SLM, a deformable mirror, or any combinationthereof. Wavefront control structure 111 may also be a phase-only SLM,which may correct the phase of a wavefront. Time sequential color system120 may be a light source for adaptive optics system 100, and may be ared light source, a green light source, and a blue light source thatemit a sequence of red, green, and blue light spaced over time. Timesequential color system 120 may include a red light emitting diode(LED), a green LED, and a blue LED that may be pulsed in sequence at anaggregate rate above 180 Hz to avoid flicker. The LEDs may also be orinclude superluminescent light emitting diodes (SLEDs), which maysupport nanofiber illuminated tools such as cannulas, cutters, forcepsand scissors. Time sequential color system 120 may be delivered by afiberoptic endoilluminator probe, and red, green, and blue light may bedelivered through the pars plana of the eye to be visualized. Wavefrontdistortion 140 of red light, green light, and blue light delivered bythe endoilluminator and reflected by the eye may be corrected by thewavefront correction system 110. In this case, wavefront distortion 140may be caused by the lens of the eye with or without a cataract, anintraocular lens, the cornea and retinal visualization system afterlight from the endoilluminator is reflected from the retina, sclera,choroid, the vitreous, scar tissue, epiretinal membrane, internallimiting membrane, hemorrhage or surgical tools. Wavefront correctionsystem 110 may use different wavefront shaping for each of the threelight colors red, green and blue emitted by time sequential color system120, which may reduce axial and lateral chromatic aberration in theadaptive optics system 100. Wavefront correction system 110 may alsoinclude processor 180, and may execute instructions on processor 180 toproduce a digital image in which at least one wavefront distortion inthe light detected by active pixel sensor arrays 130 a and 130 b ispartially or fully corrected.

Active pixel sensor arrays 130 a and 130 b may detect white light, ormay detect each of the red, green, and blue light emitted by timesequential color system 120, and may send a signal to processor 180. Inone example, processor 180 may receive one or more analog signals fromone or more of pixel sensor arrays 130 a and 130 b, among others. Theone or more analog signals from the one or more of pixel sensor arrays130 a and 130 b may be associated with detected light, which may bereflected off an interior of an eye. The analog signals may includeinformation associated with one or more frequencies of light. In anotherexample, processor 180 may receive one or more digital signals from timesequential color system 120. The one or more digital signals from theone or more of pixel sensor arrays 130 a and 130 b may be associatedwith detected light, which may be reflected off the interior of an eye.The digital signals may include information associated with one or morefrequencies of light.

Active pixel sensor arrays 130 a and 130 b may be an array of lightcapturing cells, typically with each cell representing a pixel. Activepixel sensor arrays 130 a and 130 b may contain pixels that have aphotodetector and an active amplifier. The adaptive optics system 100may include a single active pixel sensor array, may include two activepixel sensor arrays, or may include multiple active pixel sensor arrays.Light may pass through lens 160 before hitting active pixel sensorarrays 130 a and 130 b. Active pixel sensor arrays 130 a and 130 b maybe monochrome active pixel sensor arrays, may be complementarymetal-oxide-semiconductor (CMOS) sensors, or may be charge-couple device(CCD) sensors. CMOS sensors may be single chip 1080P CMOS sensors, ormay be 4K monochrome CMOS sensors. Single chip CMOS color sensorstypically make use of a Bayer filter, which is a color filter array thatmay cause lateral color spread. Active pixel sensor arrays 130 a and 130b may include a Bayer filter, or may be active pixel sensor arrayswithout a Bayer filter. Time sequential color system 120 may eliminatethe need for a Bayer filter by active pixel sensor arrays 130 a and 130b in order to provide a color digital image by allowing a red image, agreen image, and a blue image to be captured sequentially. The use ofactive pixel sensor arrays without a Bayer filter may eliminate lateralcolor spread and improve image color rendition. The use of active pixelsensor arrays without a Bayer filter may also enable chromaticaberration correction.

Amplitude-only SLM 170 may implement ROI gain control to improve dynamicrange. Amplitude-only SLM 170 may be positioned in the focal planebefore active pixel sensor arrays 130 a and 130 b. Alternatively,on-chip processor 135 on active pixel sensor arrays 130 a and 130 b mayimplement a region of interest (ROI) gain control to improve dynamicrange. Processor 135 may be an additional processor to processor 180.Processor 135 and processor 180 may be physically separate processors inthe adaptive optics system, or may be part of the same processor thatperforms several functions.

The time sequential color system 120 in adaptive optics system 100 mayinclude red light source 200, green light source 201, and blue lightsource 202, as depicted in FIG. 2. Light sources 200, 201, and 202 mayemit a sequence of red, green, and blue light spaced over time. Lightsources 200, 201, and 202 may be LEDs, and may be pulsed in sequence atan aggregate rate above 180 Hz to avoid flicker or color breakup. Lightsources 200, 201, and 202 may also be SLEDs. Light sources 200, 201, and202 may be delivered by an endoilluminator, and light may be deliveredthrough the endoilluminator fiberoptic probe through the pars plana ofthe eye to be visualized. Red light source 200 may emit red light wave210, which after being reflected by the retina or a surgical tool mayhave red-light wavefront distortion 220. Green light source 201 may emitgreen light wave 211, which after being reflected by the retina or asurgical tool may have green-light wavefront distortion 221. Blue lightsource 202 may emit blue light wave 212, which after being reflected bythe retina or a surgical tool may have blue-light wavefront distortion222. Wavefront correction system 110 may correct wavefront distortion220, 221, and 222 to corrected red wavefront 230, corrected greenwavefront 231, and corrected blue wavefront 232, by performingcorrective wavefront shaping. Wavefront shaping by wavefront correctionsystem 110 may be different for each of the wavefront distortions 220,221, and 222, and may reduce axial and lateral chromatic aberration byensuring the colors are focused to the same convergence point or imageplane.

The adaptive optics system 100 illustrated in FIG. 1 may be a componentof a vitreoretinal surgery visualization system 300, as depicted in FIG.3, to visualize eye 301 during surgery. Visualization system 300 mayinclude a surgical microscope 310, adaptive optics system 100, which mayinclude processor 180, and a digital display 330 such as a screen, ahead up display, a head mounted display, or any combination thereof.Digital display 330 may also include multiple displays. Visualizationsystem 300 may be a DAVS system, the NGENUITY® 3D Visualization System(Novartis AG Corp., Switzerland) (FIG. 4), or a camera head mounted on asurgical microscope (FIG. 5).

Visualization system 300 may include time sequential color system 120,which may be a light source for adaptive optics system 100, and may be ared light source, a green light source, and a blue light source thatemit a sequence of red, green, and blue light spaced over a period oftime. Time sequential color system 120 may be delivered byendoilluminator 305, and red, green, and blue light may be delivered byendoilluminator fiberoptic probe 306 through the pars plana of eye 301.Time sequential color system 120 may illuminate eye 301 using red,green, and blue light sources pulsed in a sequence. For example, timesequential color system 120 may illuminate eye 301 using red, green, andblue light sources pulsed in a sequence with an aggregate rate above 180Hz, which may be utilized to avoid flicker. Wavefront distortion 140 maybe caused by the lens of the eye with or without a cataract, the corneaand contact or non-contact retinal visualization system after light fromthe endoilluminator is reflected from the retina, sclera, choroid, thevitreous, scar tissue, hemorrhage or surgical tools. Wavefrontdistortion 140 present in red, green, and blue wavefronts reflected offeye 301 may be corrected by wavefront correction system 110 to reducethe effects of aberration and may improve the digital image resolutionof the digital image of eye 301 on digital display 330. Wavefrontcorrection system 110 may include processor 180. Wavefront correctionsystem 110 may execute instructions via processor 180 to produce adigital image. For example, wavefront correction system 110 may producea digital image in which wavefront distortion 140 in the light detectedby active pixel sensor arrays 130 a and 130 b is partially or fullycorrected to corrected wavefront 150.

Active pixel sensor arrays 130 a and 130 b may be an array oflight-capturing cells, typically with each cell representing a pixel,may include a Bayer filter, or may be active pixel sensor arrays withouta Bayer filter. Utilization of red, green, and blue light sources intime sequential color system 120 may allow active pixel sensor arrays130 a and 130 b to produce color images without a Bayer filter, whichmay reduce lateral color spread. Lateral color spread in color imagesproduced without a Bayer filter may be reduced compared to lateral colorspread in color images produced utilizing a Bayer filter. The reductionin lateral color spread may improve the color rendition of the digitalimage of eye 301 on digital display 330. Visualization system 300 mayinclude a single active pixel sensor array. Visualization system 300 mayinclude multiple active pixel sensor arrays. Active pixel sensor arrays130 a and 130 b may send a signal to processor 180. ROI gain control maybe implemented by active pixel sensor arrays 130 a and 130 b, and may beimplemented by an on-chip processor, or an additional amplitude-onlySLM. The amplitude-only SLM may be positioned in the focal plane beforeactive pixel sensor arrays 130 a and 130 b (not shown). ROI gain controlimplemented by active pixel sensor arrays 130 a and 130 b may improvethe dynamic range of the digital image of eye 301 on digital display330. Processor 180 may include, for example, a field-programmable gatearray (FPGA), a microprocessor, a microcontroller, a digital signalprocessor (DSP), a graphics processing unit (GPU), an applicationspecific integrated circuit (ASIC), or any other digital or analogcircuitry configured to interpret and/or execute program instructionsand/or process data.

Processor 180 may include any physical device able to store and/orexecute instructions. Processor 180 may execute processor instructionsto implement at least a portion of one or more systems, one or more flowcharts, one or more processes, and/or one or more methods describedherein. For example, processor 180 may execute instructions to producethe image of eye 301. Processor 180 may be configured to receiveinstructions from a memory medium. In one example, processor 180 mayinclude the memory medium. In another example, the memory medium may beexternal to processor 180. The memory medium may store the instructions.The instructions stored by the memory medium may be executable byprocessor 180 and may be configured, coded, and/or encoded withinstructions in accordance with at least a portion of one or moresystems, one or more flowcharts, one or more methods, and/or one or moreprocesses described herein.

A FPGA may be may be configured, coded, and/or encoded to implement atleast a portion of one or more systems, one or more flow charts, one ormore processes, and/or one or more methods described herein. Forexample, the FPGA may be configured, coded, and/or encoded to produce animage of eye 301. An ASIC may be may be configured to implement at leasta portion of one or more systems, one or more flow charts, one or moreprocesses, and/or one or more methods described herein. For example, theASIC may be configured, coded, and/or encoded to produce an image of eye301. A DSP may be may be configured, coded, and/or encoded to implementat least a portion of one or more systems, one or more flow charts, oneor more processes, and/or one or more methods described herein. Forexample, the DSP may be configured, coded, and/or encoded to produce animage of eye 301.

Although processor 180 is depicted separately from active pixel sensorarrays 130 a and 130 b, a single device may include processor 180 andactive pixel sensor arrays 130 a and 130 b. In one example, a singlecomputer system may include processor 180 and active pixel sensor arrays130 a and 130 b. In another example, a device may include integratedcircuits that may include processor 180 and active pixel sensor arrays130 a and 130 b.

Processor 180 may interpret and/or execute program instructions and/orprocess data stored in a memory medium. The memory medium may beconfigured in part or whole as application memory, system memory, orboth. The memory medium may include any system, device, or apparatusconfigured to hold and/or house one or more memory devices. Each memorydevice may include any system, any module or any apparatus configured toretain program instructions and/or data for a period of time (e.g.,computer-readable media). One or more servers, electronic devices, orother machines described may include one or more similar such processorsor memories that may store and execute program instructions for carryingout the functionality of the associated machine.

Surgical microscope 310 may display an image of eye 301, such as adigital image generated by processor 180 or another processor. Surgicalmicroscope 310 may display other information in addition to an image ofeye 301. Such other information may be generated by processor 180 oranother processor and may include graphic or textual information, suchas warnings, graphs, color coding, surgical parameters, endoscopicvideo, optical coherence tomography (OCT) images, or augmented realityinformation.

Digital display 330 may similarly display a digital image of eye 301generated by processor 180 or another processor and other informationgenerated by processor 180 or another processor. Such information mayinclude graphic or textual information, such as surgical parameters,surgical modes, flow rates, intraocular pressure, endoscopic video, OCTimages, warnings, digital images, color coding or augmented realityinformation. The information displayed on digital display 330 may notmatch that displayed on or seen using surgical microscope 310. Processor180 may reformat video made using time sequential color system 120 as alight source for display on digital display 330, which may be viewedwith circularly polarized glasses, digital oculars, or using a headmounted display.

Visualization system 300 may further contain other elements tofacilitate its uses, such as memory to store images displayed on digitaldisplay 330, electrical connections, and hardware to position and focusany lenses, such as lens 160, and to position active pixel sensor arrays130 a and 130 b.

Adaptive optics system 100 may be used as a component of the NGENUITY®DSM-1 system (Novartis AG Corp., Switzerland) in visualization system400, wherein it may replace the sensor portion as depicted in FIG. 4.For example, the adaptive optics system 100 may utilize optomechanicalfocus system 410, zoom system 420, variable working distance system 430,display system 440, and a memory medium 450. In one example, memorymedium 450 may store data that may be utilized in implementing at leasta portion of one or more systems, one or more flow charts, one or moreprocesses, and/or one or more methods described herein. In anotherexample, memory medium 450 may store instructions executable byprocessor 180 in implementing at least a portion of one or more systems,one or more flow charts, one or more processes, and/or one or moremethods described herein. As illustrated, memory medium 450 may becommunicatively coupled to processor 180. As shown, display 440 may becommunicatively coupled to processor 180.

Adaptive optics system 100 may be used as a component of a camera head510 in visualization system 500, which may further include surgicalmicroscope without oculars 520, as depicted in FIG. 5. Visualizationsystem 500 may also include processor 180, and may further includedisplay system 540, and memory medium 550. Camera head 510 may be theNGENUITY® 1.0 (Novartis AG Corp., Switzerland) camera head. In oneexample, memory medium 550 may store data that may be utilized inimplementing at least a portion of one or more systems, one or more flowcharts, one or more processes, and/or one or more methods describedherein. In another example, memory medium 550 may store instructionsexecutable by processor 180 in implementing at least a portion of one ormore systems, one or more flow charts, one or more processes, and/or oneor more methods described herein. As shown in FIG. 5, memory medium 550may be communicatively coupled to processor 180. Display 540 may also becommunicatively coupled to processor 180.

The adaptive optics system 100 to improve a digital image forvitreoretinal surgery may include image reference system 600 as depictedin FIG. 6. As shown in FIG. 6, image reference system 600 may includeimage reference 605, which may be a reference object with a known shape.Image reference 605 may be an object or surgical tool placed in eye 301.Image reference 605 may be a vitreous cutter, forceps, scissors, a pic,a scraper, a flex loop, a spatula, a MVR blade, a micro-cannula, oranother surgical tool. If the shape of image reference 605 is notalready known, it may also be placed outside of eye 301 in various posesbefore surgery commences to obtain non-aberrated image of imagereference 605. This may allow image reference 605 to be used as an imagereference when placed internally in eye 301 during surgery. Red, green,and blue light from the time sequential color system 120, which may bedelivered by endoilluminator 305, may be reflected off image reference605, may have wavefront distortion 610, and may take optical path 630.Red, green, and blue light from the time sequential color system 120 mayalso be reflected off eye 301, may have wavefront distortion 620, andmay take optical path 640. Processor 670 may analyze wavefrontdistortion 610 in the digital image of image reference 605 provided byactive pixel sensor arrays 130 a and 130 b and send instructions towavefront correction system 110 to control and correct wavefrontdistortion 610 to corrected wavefront 680 to restore the aberrated imageof image reference 605 to a non-aberrated image. These same instructionsmay be used to correct wavefront distortion 620 to corrected wavefront690. Corrected wavefront 690 may be sent to active pixel sensor arrays130 a and 130 b and may allow a digital image of eye 301 with improvedresolution. Although processor 670 is depicted separately from processor180 and active pixel sensor arrays 130 a and 130 b, they may be the sameprocessor, or part of a single physical device, such as a singlecomputer or a set of integrated circuits.

FIG. 7 presents a flow chart for a method of correcting a wavefrontdistortion to improve a digital image according to the disclosure. Instep 700, light is used to illuminate an interior of an eye duringsurgery. The light may include a sequence of red, green, and blue light,such as that emitted by time sequential color system 120 and deliveredby endoilluminator 305. The light is reflected off the interior of theeye to give a reflected wavefront of light. In step 710, the wavefrontdistortions of the reflected wavefront of light are determined. Thisstep may involve concurrent use of an image reference system, such asimage reference system 700. In step 701, light is used to illuminate animage reference during surgery. The light used to illuminate the imagereference may include red, green, and blue light, such as that emittedby time sequential color system 120 and delivered by endoilluminator305. The wavefront distortions of a reflected wavefront of lightreflected off the image reference are determined in step 711. In step720, a wavefront correction system, which may include a wavefrontcontrol structure that may be a SLM, a reflective LCoS-SLM, atransmissive LCoS-SLM, a deformable mirror, or any combination thereof,is used to correct the wavefront distortion for the reflected wavefrontof light reflected off the interior of the eye, improving the imageresolution of the digital image of the interior of the eye.

FIG. 8 presents a flow chart for a method of eliminating lateral colorspread and improving color rendition to improve a digital image forvitreoretinal surgery. In step 800, red light, green light, and bluelight, such as that emitted by time sequential color system 120, areemitted in a sequence spaced over time to illuminate an interior of aneye during surgery. The red light, the green light, and the blue lightare reflected off the interior of the eye in step 810 and may bedetected by an active pixel sensor array. The active pixel sensor arraymay be an active pixel sensor array without a Bayer filter. In step 820,a red image, a green image, and a blue image are captured sequentiallyby the active pixel sensor array without a Bayer filter. The red image,the green image, and the blue image may be integrated in the viewer'svisual cortex, or may be reformatted by an image processor to bedisplayed on an OLED display or head mounted display, to give a colorimage of the interior of the eye with lateral color spread eliminated.This method may improve the color rendition of the digital image of theinterior of the eye.

Computer system 900 is depicted in FIG. 9. Computer system 900 mayinclude a processor 910, a volatile memory medium 920, a non-volatilememory medium 930, and an input/output (I/O) device 940. Volatile memorymedium 920, non-volatile memory medium 930, and I/O device 940 may becommunicatively coupled to processor 910.

The term “memory medium” may mean a “memory”, a “storage device”, a“memory device”, a “computer-readable medium”, and/or a “tangiblecomputer readable storage medium”. For example, a memory medium mayinclude, without limitation, storage media such as a direct accessstorage device, including a hard disk drive, a sequential access storagedevice, such as a tape disk drive, compact disk (CD), random accessmemory (RAM), read-only memory (ROM), CD-ROM, digital versatile disc(DVD), electrically erasable programmable read-only memory (EEPROM),flash memory, non-transitory media, or any combination thereof. As shownin FIG. 9, non-volatile memory medium 930 may include processorinstructions 932. Processor instructions 932 may be executed byprocessor 910. In one example, one or more portions of processorinstructions 932 may be executed via non-volatile memory medium 930. Inanother example, one or more portions of processor instructions 932 maybe executed via volatile memory medium 920. One or more portions ofprocessor instructions 932 may be transferred to volatile memory medium920.

Processor 910 may execute processor instructions 932 in implementing atleast a portion of one or more systems, one or more flow charts, one ormore processes, and/or one or more methods described herein. Forexample, processor instructions 932 may be configured, coded, and/orencoded with instructions in accordance with at least a portion of oneor more systems, one or more flowcharts, one or more methods, and/or oneor more processes described herein. Although processor 910 isillustrated as a single processor, processor 910 may be or includemultiple processors. One or more of a storage medium and a memory mediummay be a software product, a program product, and/or an article ofmanufacture. For example, the software product, the program product,and/or the article of manufacture may be configured, coded, and/orencoded with instructions, executable by a processor, in accordance withat least a portion of one or more systems, one or more flowcharts, oneor more methods, and/or one or more processes described herein.

Processor 910 may include any suitable system, device, or apparatusoperable to interpret and execute program instructions, process data, orboth stored in a memory medium and/or received via a network. Processor910 further may include one or more microprocessors, microcontrollers,digital signal processors (DSPs), application specific integratedcircuits (ASICs), or other circuitry configured to interpret and executeprogram instructions, process data, or both.

I/O device 940 may include any instrumentality or instrumentalities,which allow, permit, and/or enable a user to interact with computersystem 900 and its associated components by facilitating input from auser and output to a user. Facilitating input from a user may allow theuser to manipulate and/or control computer system 900, and facilitatingoutput to a user may allow computer system 900 to indicate effects ofthe user's manipulation and/or control. For example, I/O device 940 mayallow a user to input data, instructions, or both into computer system900, and otherwise manipulate and/or control computer system 900 and itsassociated components. I/O devices may include user interface devices,such as a keyboard, a mouse, a touch screen, a joystick, a handheldlens, a tool tracking device, a coordinate input device, or any otherI/O device suitable to be used with a system.

I/O device 940 may include one or more buses, one or more serialdevices, and/or one or more network interfaces, among others, that mayfacilitate and/or permit processor 910 to implement at least a portionof one or more systems, processes, and/or methods described herein. Inone example, I/O device 940 may include a storage interface that mayfacilitate and/or permit processor 910 to communicate with an externalstorage. The storage interface may include one or more of a universalserial bus (USB) interface, a SATA (Serial ATA) interface, a PATA(Parallel ATA) interface, and a small computer system interface (SCSI),among others. In a second example, I/O device 940 may include a networkinterface that may facilitate and/or permit processor 910 to communicatewith a network. I/O device 940 may include one or more of a wirelessnetwork interface and a wired network interface. In a third example, I/Odevice 940 may include one or more of a peripheral componentinterconnect (PCI) interface, a PCI Express (PCIe) interface, a serialperipheral interconnect (SPI) interface, and an inter-integrated circuit(I2C) interface, among others. In a fourth example, I/O device 940 mayinclude circuitry that may permit processor 910 to communicate data withone or more sensors. In a fifth example, I/O device 940 may facilitateand/or permit processor 910 to communicate data with one or more of adisplay 950 and adaptive optics system 100, among others. As shown inFIG. 9, I/O device 940 may be coupled to a network 970. For example, I/Odevice 940 may include a network interface.

Network 970 may include a wired network, a wireless network, an opticalnetwork, or any combination thereof. Network 970 may include and/or becoupled to various types of communications networks. For example,network 970 may include and/or be coupled to a local area network (LAN),a wide area network (WAN), an Internet, a public switched telephonenetwork (PSTN), a cellular telephone network, a satellite telephonenetwork, or any combination thereof. A WAN may include a private WAN, acorporate WAN, a public WAN, or any combination thereof.

Although FIG. 9 illustrates computer system 900 as external to adaptiveoptics system 100, adaptive optics system 100 may include computersystem 900. For example, processor 910 may be or include processor 180.

FIGS. 10A-10C illustrate examples of medical system 1000. As shown inFIG. 10A, medical system 1000 may include adaptive optics system 100. Asillustrated in FIG. 10B, medical system 1000 may include adaptive opticssystem 100 and computer system 900. Adaptive optics system 100 may becommunicatively coupled with computer system 900. As shown in FIG. 10C,medical system 1000 may include adaptive optics system 100, which mayinclude computer system 900.

Adaptive optics system 100 may be used as a component of medical system1100, as shown in FIG. 11. Medical system 1100 may include adaptiveoptics system 100. Medical system 1100 may include computer system 900.Surgeon 1110 may view a digital image of eye 1301 of patient 1120 onmicroscope integrated display (MID) 1130, display 1150, or anycombination thereof. MID 1130, display 1150, or any combination thereof,may display an image of eye 1301 in which at least one wavefront oflight reflected off the interior of eye 1301 is fully or partiallycorrected. The digital image of eye 1301 may have improved imageresolution, improved image color rendition, improved image dynamicrange, or any combination thereof, compared to a digital image of theeye captured without adaptive optics system 100. Medical system 1100 mayinclude a processor; at least one active pixel sensor array coupled tothe processor; a wavefront correction system coupled to the processor;an image reference system; and a memory medium, such as those inadaptive optics system 100 or image reference system 600. The memorymedium may be coupled to the processor, and may include instructionsthat when executed by the processor, cause the medical system to utilizethe image reference system to determine a wavefront distortion of areflected wavefront of light reflected off the interior of eye 1301 ofpatient 1120. The memory medium may also include instructions that whenexecuted by the processor, cause the medical system to utilize thewavefront correction system to correct the wavefront distortion of thereflected wavefront of light reflected off the interior of eye 1301.

Adaptive optics system 100, visualization system 300, visualizationsystem 400, visualization system 500, image reference system 600,computer system 900, and medical system 1000, medical system 1100, andcomponents thereof may be combined with other elements of visualizationtools and systems described herein unless clearly mutually exclusive.For instance, the image reference and processor in image referencesystem 600 may be used with other visualization systems describedherein.

The above disclosed subject matter is to be considered illustrative, andnot restrictive, and the appended claims are intended to cover all suchmodifications, enhancements, and other embodiments which fall within thetrue spirit and scope of the present disclosure. For example, althoughan adaptive optics system is most commonly needed to improve a digitalimage during vitreoretinal surgery, if it were useful in anotherprocedure, such as a purely diagnostic procedure not otherwiseconsidered to be surgery, the systems and methods described herein maybe employed.

1. An adaptive optics system comprising: at least one active pixelsensor array operable to detect light and send a signal to a processor;and a wavefront correction system comprising at least one wavefrontcontrol structure and the processor and operable to: executeinstructions on the processor to produce a digital image in which atleast one wavefront distortion in the light detected by the active pixelsensor array is partially or fully corrected.
 2. The adaptive opticssystem of claim 1 further comprising a plurality of active pixel sensorarrays.
 3. The adaptive optics system of claim 1, wherein the wavefrontcontrol structure comprises a spatial light modulator (SLM). a liquidcrystal on silicon spatial light modulator (LCoS-SLM), a transmissiveLCoS-SLM, a reflective LCoS-SLM, a deformable mirror, or any combinationthereof.
 4. The adaptive optics system of claim 1, wherein the wavefrontcontrol structure comprises a phase-only SLM.
 5. The adaptive opticssystem of claim 1, wherein the digital image is displayed on a digitaldisplay, a screen, a head up display, a head mounted display, or anycombination thereof.
 6. The adaptive optics system of claim 1, whereinthe adaptive optics system is a component of NGENUITY®.
 7. The adaptiveoptics system of claim 1 wherein the active pixel sensor array is acomplementary metal-oxide-semiconductor (CMOS) monochrome sensor, a 4Kmonochrome CMOS sensor, a 1080P monochrome CMOS sensor, or anycombination thereof.
 8. The adaptive optics system of claim 1, furthercomprising an on-chip processor on the active pixel sensor arrayoperable to implement a region of interest (ROI) gain control.
 9. Theadaptive optics system of claim 1, further comprising an amplitude-onlySLM to implement a ROI gain control on the active pixel sensor array.10. The adaptive optics system of claim 1, further comprising an imagereference system.
 11. The adaptive optics system of claim 10, whereinthe image reference system comprises an image reference that is asurgical tool placed in an eye.
 12. The adaptive optics system of claim11, wherein the surgical tool is a vitreous cutter, forceps, scissors, apic, a scraper, a flex loop, a spatula, a micro-vitreoretinal (MVR)blade, a micro-cannula, or any combination thereof.
 13. The adaptiveoptics system of claim 1, further comprising: a time sequential colorsystem comprising a red light source, a green light source, and a bluelight source operable to emit a sequence of red, green, and blue lightspaced over a period of time; and wherein the at least one active pixelsensor array is operable to detect each of the red, green, and bluelight, and send a signal to the processor.
 14. The adaptive opticssystem of claim 13, wherein the time sequential color system comprises ared light emitting diode (LED), a green LED, and a blue LED pulsed in asequence with an aggregate rate of 180 Hz or above.
 15. The adaptiveoptics system of claim 13, wherein the time sequential color systemcomprises a red superluminescent light emitting diode (SLED), a greenSLED, and a blue SLED pulsed in sequence with an aggregate rate of 180Hz or above.
 16. The adaptive optics system of claim 13, wherein thetime sequential color system is delivered by an endoilluminator.
 17. Theadaptive optics system of claim 13, wherein the active pixel sensorarray is an active pixel sensor array without a Bayer filter.
 18. Theadaptive optics system of claim 13, wherein the active pixel sensorarray captures sequentially a red image, a green image, and a blueimage.
 19. A medical system, comprising: a processor; at least oneactive pixel sensor array coupled to the processor; a wavefrontcorrection system coupled to the processor; an image reference system;and a memory medium that is coupled to the processor and that includesinstructions, when executed by the processor, cause the medical systemto: utilize the image reference system to determine a wavefrontdistortion of a reflected wavefront of light reflected off an interiorof an eye of a patient; and utilize the wavefront correction system tocorrect the wavefront distortion of the reflected wavefront of lightreflected off the interior of the eye.
 20. A method of correcting awavefront distortion to improve a digital image, the method comprising:using an image reference system to determine a wavefront distortion of areflected wavefront of light reflected off the interior of the eye; andusing a wavefront correction system to correct the wavefront distortionof the reflected wavefront of light reflected off the interior of theeye.
 21. A method of eliminating lateral color spread and improvingcolor rendition to improve a digital image, the method comprising: usinga time sequential color system emitting a sequence of red light, greenlight, and blue light spaced over time to illuminate an interior of aneye; detecting a red wavefront of light, a green wavefront of light, anda blue wavefront of light reflected off the interior of the eye using atleast one active pixel sensor array without a Bayer filter; capturing ared image, a green image, and a blue image of the interior of the eyesequentially using the active pixel sensor array without a Bayer filter;and integrating the red image, the green image, and the blue image inthe viewer's visual cortex, or reformatting the red image, the greenimage, and the blue image for an organic light-emitting diode (OLED)display, to give a color image of the interior of the eye with lateralcolor spread eliminated.